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FOREWORD 


The  effectivene;  s  of  planning  and  conducting  an  R&D  program 
can  in  part  be  measured  by  the  ultimate  use  which  program 
results  find  in  subsequent  technology  and  system  applications. 
There  have  been  several  efforts  to  measure  i.uch  effectiveness. 
They  include  the  Department  of  Defense  "Project  Hindsight" 
(October  1969)  and  the  National  Science  Foundation  "Traces" 
study  (1968).  Tracing  the  origins  of  technology  and  of  systems 
was  the  technique  used  in  these  studies  in  trying  to  evaluate 
earlier  research  and  technology  planning  and  program  effective¬ 
ness.  The  question  of  how  well  one  might  expect  to  trace  the 
research  results  comprising  origins  of  technology  (and  of 
systems)  in  being  has  a  bearing  on  interpretation  of  the  results 
of  any  such  study.  A  high  probability  of  success  in  tracing 
origins  would  mean  a  high  confidence  in  study  results.  A  low 
probability  would  mean  a  low  confidence  in  study  results. 

The  objective  of  this  study  was  to  evaluate  the  probability  of 
success  in  tracing  origins  of  technology  and  of  systems.  A 
mathematical  model  from  statistical  communication  theory  and 
data  from  US  Air  Force  research  planners  were  used  to  obtain 
quantitative  results. 

The  study  was  done  by  the  author  while  serving  in  an  earlier 
assignment  as  Director  of  Plans  at  the  Office  of  Aerospace 
Research  (OAR)  in  Arlington,  Virginia.  This  work  was  originally 
reported  in  the  Fourth  Office  of  Aerospace  Research  Seminar  on 
R&D  Coupling  and  Information  Transfer  under  Mr.  Alexander 
Hoshovsky,  Directorate  of  Scientific  and  Technical  Information, 
on  13  June  1969.  It  is  being  published  at  the  request  of 
Headquarters  OAR. 

This  report  has  been  reviewed  and  is  approved  for  publication. 


ERNEST 'j>\DAVIS,  JR.  ,  Ollonel,  USAF 
Commander,  Office  of  Research  Analyses 


ABSTRACT 


A  simple  model  from  statistical  communication  theory  is  used  to 
evaluate  the  probability  of  success  in  tracing  research  results 
which  comprise  origins  of  technology.  The  model  has  also  been 
used  to  evaluate  the  probability  of  success  in  tracing  research 
and  technology  origins  of  systems.  A  by-product  of  this  study 
is  the  use  of  the  same  model  for  evaluating  the  probability  of 
success  of  forecasting  applications  of  research  results  and  of 
technology  advances.  l>se  of  the  model  has  permitted  the  con¬ 
clusion  that  certain  fundamental  limitations  to  successful 
tracing  and  forecasting  exist.  These  limitations  are  analogous 
to  well-known  physical  limitations  in  successful  electrical 
communication  —  bandwidth  and  noise.  Bandwidth  and  noise  are 
closely  related  to  the  classification  systems  for  the  originating 
categories  (bandwidth)  and  for  the  receiving  categories  (noise). 
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INTRODUCTION 
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Planning 

Planning  in  any  context  involves  two  essential  concepts: 
assessing  the  future  and  making  provision  for  the  future.  A  more 
formal  definition  (Reference  1)  is  as  follows: 

"Planning  is  an  analytical  process  which  encompasses 
an  assessment  of  the  future,  the  determination  of  desired  objectives 
in  the  context  of  that  future,  the  development  of  alternative  courses 
of  action  to  achieve  such  objectives,  and  the  selection  of  a  course 
(or  courses)  of  action  from  among  these  alternatives." 

Planning  is  closely  related  to  another  concept  —  control  — • 
in  practical  situations.  Since  control  is  the  process  of  measuring 
operating  performance  and  seeing  that  it  conforms  to  plans  as 
closely  as  possible,  control  is  necessary  if  we  are  to  receive  any 
feedback  regarding  the  progress  of  a  plan. 

Since  planning  is,  by  definition,  concerned  with  the  future, 
it  is  necessarily  Involved  with  the  uncertainty  and  probabilities 
of  future  circumstances.  Ultimate  inclusion  of  a  technology 
advance  in  some  practical  application  as  a  part  of  a  larger  system 
may  Involve  a  five  to  tan  year  time-lag  because  of  the  .lature  of 
the  process  of  system  design.  For  example,  assume  that  a  new  method 
for  augmenting  thrust  for  vertical  takeoff  end  lending  (VTOL)  aircraft 
has  been  perfected  in  year  1.  if  the  method  were  incorporated 


immediately  into  the  design  of  a  new  aircraft,  it  would  not 
appear  in  use  until  the  aircraft  had  been  through  the  complete 
design,  development,  and  manufacturing  process  —  probably  in  five 
or  more  years  beyond  the  reference  year.  Thus,  for  technology-to- 
systems  application,  five  or  more  years  can  be  involved  in  the 
transition  from  technology  concept  to  system  application.  The 
transition  interval  between  the  availability  of  a  research  result 
and  its  use  in  technology  has  been  estimated  to  require  at  least 
the  same  or,  realistically,  an  even  longer  time  interval,  on  the 
average.  Thus,  we  may  conclude  that  at  least  ten  years  is  involved 
in  going  from  research  to  systems  applications. 

Research  and  Development  Planning 

In  recent  years,  long-range  planning  has  been  receiving 
increasing  attention  and  emphasis.  This  need  for  planning  is 
especially  apparent  in  research  and  development.  The  size  of  even 
the  research  portion  of  the  overall  military  research  and  develop¬ 
ment  budget  has  increased  until  it  now  occupies  a  substantial  claim 
on  the  attentions  of  decision-makers  at  many  levels.  Planning  of 
research  within  the  military  services  must  clearly  involve  planning 
for  the  accomplishment  of  research  with  promise  for  utility  in 
military  technology.  This  promise  of  utility  is  now  characterized 
by  the  expression  "relevance." 

Thus,  for  each  research  effort,  the  decision-maker  must 
have  identified  the  technology  or  system  for  which  the  effort  has 
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future  releva. ice.  In  parallel  fashion,  the  technology  planner 
must  identify  the  military  weapon  or  weapons,  or  possibly  the 
military  function's)  for  which  each  technology  provides  support. 
Forecasting  in  Research  and  Development  Planning 
In  recent  years  forecasting,  and  especially  technological 
forecasting,  has  attracted  considerable  attention  as  a  potential 
means  of  assisting  in  the  planning  processes  both  in  industry 
and  in  the  military  development  agencies,  since  it  is  a 
methodology  for  assessing  the  future.  Technological  forecasting 
has  become  an  expanding  discipline  to  which  an  increasing  number 
of  books  and  periodicals  are  now  being  devoted.  Forecasting  has 
been  divided  into  the  normative,  or  mission-oriented,  and  the 
exploratory,  or  capability-oriented,  approaches.  Normative  fore¬ 
casting  is  oriented  toward  consideration  of  present  needs  and 
decisions  as  deduced  from  a  desired  future  situation  while 
exploratory  forecasting  uses  the  past  and  present  to  deduce  future 
capabilities.  The  correspondence  between  the  two  essential 
planning  concepts  and  the  available  forecasting  approaches 
i«  shown  here: 

Planning  Concept:  Related  Forecasting  Approach 

1.  Assessing  the  future.  Exploratory  foretasting. 

2.  Making  provision  for  Normative  forecasting, 

that  future  (determina¬ 
tion  of  objectives, 

development  of  alterna¬ 
tives  to  achieve 
objectives). 
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iluut,  on  the  one  hand  there  is  exploratory  forecasting  —  an 
attempt  to  predict  the  technological  state-of-the-art  that  will 
or  might  be  in  the  future.  Most  laymen  assume  that  all  of  fore- 

t 

cashing  is  this  kind  of  forecasting.  The  second  aspect ,  normative 
forecasting,  includes  the  organized  attempts  to  allocate  money, 
manpower,  and  other  resources  that  might  affect  the  creation  of 
tomorrow’s  technological  state-of-the-art.  Exploratory  technological 
forecasting  starts  from  today's  assured  basis  of  knowledge  and  is 
oriented  toward  the  future,  while  normative  technological  forecasting 
first  assesses  future  goals,  predicted  technology  capability,  and 
missions,  and  works  back  to  the  present.  "The  full  potential  of 
technological  forecasting  is  realized  only  where  exploratory  and 
normative  efforts  are  joined  in  an  iterative  or  ultiiiiately  in  a 
feedback  cycle  "  (Reference  2).  An  understanding  oC  the  informa¬ 
tion  flow  in  these  processes  would  be  helpful  in  improving  their 
effectiveness. 

Research  and  Development  Evaluation 

Evaluation  of  research  and  development  programs  from  the  point 
of  view  of  their  conformity  to  planned  performance  is  a  difficult 
task.  Their  conformity  to  planned  performance  can  be  measured 
by  their  contributions  to  actual  subsystems  or  systems.  Other 
measures  of  this  conformity  are  their  contributions  in  the  form  of 
providing  alternative  potential  solutions  to  a  technical  problem, 
only  one  of  which  could  be  chosen,  and  their  contributions  to  the 
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avoidance  of  technological  surprise.  A  common  denominator  in 
these  processes  is  the  transfer  of  information.  A  measure  of 
the  success  of  information  transfer  and  use  in  the  R&D  evaluation 
process  would  be  useful  in  establishing  the  reliability  of  the 
evaluation  itself. 
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II. 


A  MATHEMATICAL  MODEL  FOR  INFORMATION  TRANSFER 


A  knowledge  of  the  mechanism  of  information  transfer  is 
Important  in  research  and  development  planning  as  well  as  in  its 
evaluation.  An  established  discipline  which  provides  a  tool  for 
analysis  is  the  statistical  theory  of  communication.  It  contains 
methods  for  the  study  of  the  statistical  problems  encountered  in 
all  types  of  communications.  Information  theory  is  a  part  of  this 
broad  discipline. 

The  communication  of  information  is  generally  of  a  statistical 
nature.  Information  theory  is  the  study  of  simple,  ideal  statistical 
communication  models.  The  objective  of  information  theory  is  to 
define  different  types  of  information  sources  and  channels,  and 
to  devise  statistical  parameters  describing  their  operations. 

Statistical  communication  theory  is  generally  regarded  as 
having  been  founded  by  Shannon  (1948)  and  Wiener  (1949).  These 
scientists  conceived  of  the  communication  situation  as  one  in 
which  a  signal  chosen  from  a  specified  class  is  to  be  transmitted 
through  a  channel,  but  the  output  of  the  channel  is  not  completely 
determined  by  the  input.  Instead,  the  channel  is  described  statis¬ 
tically  by  giving  a  probability  distribution  over  the  set  of  all 
possible  outputs  for  each  permissible  input.  At  the  output  of  the 
channel,  a  received  signal  is  observed,  and  then  a  decision  is  made, 
the  objective  of  the  decision  being  to  identify  as  closely  as  possible 
some  property  of  the  input  signal. 
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The  applicability  of  the  Shannon  model  to  the  communication 
situation  existing  between  research  and  technology  is  appealing. 

The  use,  in  technology,  of  scientific  information  generated  through 
research  is  impredictable.  Hence,  as  in  the  Shannon  formulation, 
a  signal  (a  research  result)  from  a  specified  class  of  signals 
(research  discipline  or  Defense  Research  Sciences  subelement)  is 
to  be  transmitted  through  a  channel  (represented  by  the  flow  of 
research  information  to  technological  application) ,  but  the  output 
of  this  channel  cannot  be  uniquely  deduced  from  the  input . 

Information  Theory 

Information  is  the  entity  which  makes  the  difference  between 
knowing  and  not  knowing,  between  being  faced  with  a  number  of 
possibilities  and  knowing  the  one  that  actually  prevails.  To 
define  it  quantitatively,  we  consider  a  simple  case  of  choice  be¬ 
tween  n  possibilities.  We  hope  that  other  cases  will  be  reducible 
to  this  simple  one  or  that  suitable  generalization  of  our  definition 
will  prove  possible. 

If  we  have  reason  to  be  sure  that  a  coming  Event  will  be  one 
out  of  n  equiprobable  Events,  the  probability  of  each  of  the  possibil 
ities  is  taken  (by  ordinary  probability  calculations)  tc  be  1/n. 
Suppose  a  situation  where  initially  the  outcome  may  be  one  of  n^. 

We  now  acquire  information  which  reduces  the  possibilities  to 

n  (<  n  );  then  further  information  reducing  them  to  n  (<  n  ) ; 

2  1  3  2 
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and  finally  to  one  possibility  (nc  ■  1;  pc  ■  1.0).  The  "effect" 
or  value  of  the  information  acquired  at  each  step  may  be  repre¬ 
sented  by  the  fraction  that  will  convert  p^l/nj)  to  p2(l/n2), 
i.e. ,  l/n2  divided  by  l/n^  which  equals  n^/n-,. 

The  "effect"  required  to  increase  the  original  probability  « 

of  1/n.  to  1.0  in  a  single  step  is  1.0  divided  by  l/n^  which 
equals  n^ ,  and  the  product  of  the  information  ratios  over  the 
steps  gives  the  same  result.  This  effect  is  shown  in  Figure  1. 


EFFECT  OF  INFORMATION 


Steps 

1 

2 

3 

c 

Number  of 
Possibilities 

nl 

n2 

n3 

nc  *  1 

Equivalent 

Probability 

1/nj 

l/n2 

i/na 

1.0 

"Effect"  of 
Information 

nj/n2 

n2/n3 

n3/l 

or  (in 

probabilities) 

P2/P! 

P3/P2 

i.o/p3 

Or  with  Numerical  Examples: 

Number  of 
Possibilities 

10 

6 

4 

1 

« 

Equivalent 

Probability 

1/10 

1/6 

1/4 

1.0 

5 

"Effect"  of 
Information 

10/6 

6/4 

4/1 

FIGURE  1 


Up  to  this  point,  we  have  a  straightforward  mathematical 
calculation  in  which  only  the  simplest  notion  of  probability  has 
been  used.  In  the  Informational  calculus  the  step  has  been  taken 
of  using  logarithmic  values  instead  of  the  fractional 
values.  This  means  that  the  effects  of  information  can  be  added 
instead  of  being  multiplied,  which  simplifies  calculation.  The  value 
"log  n"  can  be  given  two  slightly  different  interpretations  which, 
for  later  purposes,  should  be  distinguished. 

Use  of  Logarithms: 
n  n  n 

1  2  3  10  6  4 

-  X - X  -  -  n  ,  OR  -==-  X  —  X  —  -  10 

n  n  1  1  6  4  1 

2  3 

Logarithms  could  be  used  for  calculating  these  values  and 
their  products,  i.e. : 

n  n  n 

1  2  3 

log  -  X  -  X  -  ■  log  n 

n  n  1  1 

2  3 

or  (log  n  -  log  n  )  (log  n  -  log  n  )  +  (log  n  -  log  1)  •  log  n 

1  2  2  3  3  1 

or  in  terms  of  probabilities: 

1  1  1  1  1 
log  — —  -  log  +  log - -  log  ——  +  log -  -  log  1 

P1  P2  z  P3  P3 

-  log  -  -log  p^ 
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A  further,  though  not  essential  step,  is  usually  taken  to  pro¬ 
vide  a  convenient  unit  of  measurement.  The  information  required 
to  reduce  n  possibilities  to  n/2  will  be  log  (n  *  n/2)  ■  log  2. 

If  instead  of  common  logarithms  (to  base  10)  we  use  logarithms  to 
base  2,  since  21  *  2,  log22  ■  1.0,  and  this  gives  as  the  unit  of 
information  that  which  halves  the  possible  outcomes  or  doubles 
the  probability  of  each  outcome.  This  unit  is  called  a  binary  unit  or 
"bit."  Thus,  in  summary,  when  n  -  the  number  of  equiprobable 
possibilities : 

Total  Information  Required  ■  log2n  or  -log2p2. 

Information  to  reduce  n^  possibilities  to 

n2  -  log^j  -  log2n2  or  (-log2p1)  -  (-log2p2). 

Unit  of  Information  (bit)  -  Information  to  reduce  the  possi¬ 
bilities  by  one-half,  or  increase  the  probability  by  a  factor  of  two. 

Interpretations  of  Log  n: 

1.  If  there  are  n  equiprobable  outcomes,  each  has  an  initial 
probability  of  1/n,  so  that  log  n  -  log  1/p  or  -log  p.  In  this 
sense,  it  is  the  amount  of  information  required  to  increase  the 
probability  of  any  one  outcome  to  1.0.  Since  p  is  never  greater  than 
1.0,  log  p  is  always  negative,  but  with  the  negative  sign,  yields 

a  positive  value. 

2.  When  the  n  outcomes  are  equiprobable,  each  will  require  in¬ 
formation  of  log  n  to  raise  it  to  certainty.  Therefore,  log  n  is  also 
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the  average  information  required  over  the  whole  set  of  possible 
outcomes,  and  in  this  sense,  can  be  taken  as  representing  the  degree 
of  uncertainty  that  exists  before  further  information  is  obtained. 

The  distinction  between  information  in  its  everyday  sense 
and  Information  as  a  calculated  value  (for  which  I  shall,  in  the  future, 
use  a  capital  letter)  can  be  indicated  by  a  simple  example.  Suppose 
I  am  told  that  "first  right,  second  left,  and  straight  ahead"  will 
take  me  to  the  Town  Hall,  and  that  these  directions  are,  in  fact,  adequate 
to  get  me  there.  If  my  guide  adds  the  remark:  "you  go  past  the  main 
post  office,"  this  is  an  extra  piece  of  information  in  the  everyday 
sense,  but  in  Informational  terms,  it  has  no  Information  value  since 
I  already  have  enough  Information  to  reach  my  destination  with  certainty. 
Put  more  formally,  these  principles  apply: 

INFORMATION  AS  A  CALCULATED  VALUE 

1.  No  element  (sentence,  perception,  symbol,  etc.)  can  be 
assigned  an  Information  value  except  in  relation  to  a  definite  context 
of  possibilities. 

2.  It  follows  that  the  same  or  similar  elements  in  different 
contexts  may  have  different  Information  values.  The  measure  of 
information  is  in  no  sense  a  description  of  the  element  as  a  piece  of 
information  in  the  everyday  sense. 

In  a  set  of  equip robable  items,  the  Information  (log  n)  required 

2 

to  raise  each  item  to  certainty  is  also  the  average  requirement  over 
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all  the  items.  When  items  are  not  equiprobable ,  the  Information 
requirement  will  vary  between  items,  as  in  the  set  of  four  that 
follows,  where  p  is  the  prior  probability.  This  situation  is 
illustrated  in  Figure  2. 


UNEQUAL  PROBABILITIES 


Po 

Item 

No. 

Requirement 

Ratio  of  Gain 

Log  Ratio  » 

-  LoVo 

0.1 

1 

10:1 

3.322 

0.2 

2 

5:1 

2.322 

0.3 

3 

10:3 

1.737 

0.  A 

A 

5:2 

1.322 

FIGURE  2 


In  such  cases  a  simple  average  will  not  represent  the  average 
expectation  of  Information  since  we  must  take  into  account  that, 
for  example.  Item  1  will  occur  only  once  to  every  four  occurrences 
of  Item  A.  A  weighted  average  could  be  obtained  by  multiplying  each 
gain  by  a  representative  number  of  occurrences,  e.g.,  1,  2,  3,  A, 
and  then  averaging  overall  occurrences.  But  this  is  the  same  as 
multiplying  each  gain  by  the  probability  of  its  occurrence.  Since  the 
sum  of  probabilities  is  always  1.0,  the  weighted  average  is  then  given 
by  addition. 


12 


THE  WEIGHTED  AVERAGE 


item 

Required 

No. 

Information 

X 

p  -  -p  log  p 

o  2 

1 

3.322  (log  10) 

2 

0.1 

0.3322 

2 

2.322  (log  5) 

2 

0.2 

0.4644 

3 

1.737  (log  10  -  log  3) 

2  2 

0.3 

0.5211 

4 

1.322  (log  o  -  log  2) 

2  2 

0.4 

0.5288 

E  -  p  log  p  =  1.8465 

2 

18.465 

Mean 

(-10)  - 

1.8465 

FIGURE  3 


The  quantity  -p  log  p,  usually  symbolized  H,  can  be  calculated 


represents  the  uncertainty  of  a  set  of  items  or  categories,  and  also 

the  average  expectation  of  Information  from  them  if  they  are  disclosed. 

It  is  called  entropy  because  of  its  formal  resemblance  to  this  concept 

in  statistical  mechanics.  Its  maximum  value  is  log  n  when  n  items 

2 

are  equiprobable.  It  can  be  interpreted  in  another  way  by  restoration 
of  the  original  (nonlogarithmic)  values.  The  value  H  -  1.846  above  is 
the  log^  of  3.59,  so  that  a  set  of  items  with  these  variations  of 
probability  is,  in  effect,  equated  with  a  set  of  3.59  items. 
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Entropy  and  Information  of  Combined  Systems 

Thus  far,  information  as  a  calculated  value  has  only  been 
applied  to  a  single  set  of  items.  The  definition  can  also  be 
used  in  analyzing  actual  communication  systems.  Such  systems 
have  three  essential  parts.  In  an  electrical  communication 
system,  these  are  the  transmitter,  medium,  and  receiver.  (Reference 
3).  Generalized  communication  systems  have  analogous  parts:  the 
information  source,  the  information  channel,  and  the  information 
destination.  In  this  study,  I  am  concerned  with  three  such 
information  systems.  They  are  research- to-technology ,  technology- 
to-systems,  and  research- to-sys terns . 

Let  X  be  a  system  assuming  states  x^  and  let  Y  be  a  system 
assuming  states  y^ ;  we  have  no  direct  access  to  X,  but  can 
observe  Y.  How  much  information  on  X  can  we  obtain  by  observing 
Y? 

Part  of  the  answer  is  obvious  immediately;  we  can  obtain 
no  information  at  all  if  there  is  no  communicatiori  between  X  and 
Y,  so  that  X  and  Y  are  independent.  However,  if  X  and  Y  are  inter¬ 
dependent,  i.e. ,  there  is  some  sort  of  communication  between  them, 
then  we  can  evidently  obtain  some  information  on  X  by  observing  Y. 
As  a  matter  of  fact,  this  is  the  only  way  of  obtaining  information 
on  any  system:  any  kind  of  Information  must  ultimately  enter  the 
observer  by  sensors  (the  five  senses  if  the  observer  is  human). 


and  from  the  states  of  this  sensor  system  information  is  gained 
on  the  state  of  the  system  that  acted  on  the  sensors  of  the 
observer  via  some  communication  channel.  Thus ,  we  havo,  the 
scheme  in  which  X  acts  on  Y  via  a  communication  channel,  and  the 
state  of  Y  is  being  observed  via  an  observation  channel. 


A  MEASURE  OF  ASSOCIATION  -  JOINT  OCCURRENCES 


c2 

n2 

2 

3 

A 

B 

nl 

1 

A 

- 

1 

B 

1 

1 

2 

C 

1 

1 

FIGURE  4 

H  is  an  es&i  .;tial  step  towards  the  calculation  of  I  (x;  y), 
a  measure  of  association  between  two  sets  of  categories,  in  the 
immediate  instance,  categories  of  symbols.  Consider  the  five  equi- 
probable  codings,  with  three  types  of  symbols  in  Cj  and  two  In  C2 , 
shown  in  Figure  4. 
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Figure  4  is  a  matrix  shoving  the  numbers  of  joint  occur¬ 
rences  of  tvo  symbols,  one  an  input  symbol  (from  C  )  and  the  other 

1 

an  output  symbol  (from  C  ).  Figure  5  shows  the  probabilities  of 

2 

the  joint  occurrences.  These  are  derived  from  Figure  4  by  dividing 

each  matrix  element  by  n  (a  5),  the  sum  of  the  matrix  elements. 

In  Figure  6,  a  joint  information  matrix  with  each  element  calculated 

from  the  relation  H  *  -p  log  d  is  shown. 

2' 

A  MEASURE  OF  ASSOCIATION  -  JOINT  PROBABILITIES 


C 


C2 


'  pj 

Pi 

0.4 

0.6 

A 

B 

0.2 

A 

- 

0.2 

0.4 

B 

0.2 

0.2 

0.4 

C 

0.2 

0.2 

FIGURE  5 


.529  +  .442 


H(2)  ■  0.971  blts/j-th  state  (average) 

H(l»  2)  -  -I  Z  p  log  p  (4) 

ij  ij  2  iJ 

-  <0  +  .464)  +  (.464  +  .464) 

+  (.464  +  .464) 


H(l,  2) 
1(1;  2)  - 

id;  2)  « 


“  2.320  bits/ij-th  state  (average) 

1.522  +  0.971  -  2.320,  from  Eq  (1). 
0.173  bits/ij-th  state  (average). 


1(1;  2)  can  never  be  larger  than  the  smaller  of  H(l)  or 
H(2).  Therefore,  in  this  case,  it  could  never  be  larger  than 
H(2)  ■  0.971.  Because  of  the  lack  of  a  one-to-one  correspondence 
between  input  and  output  signals,  1(1;  2)  in  this  case  is  considerably 
smaller  than  its  potential  maximum  of  H(2)  *  0.971. 

1(1;  2)  can  also  be  calculated  by  another  method  which  is 
less  convenient  in  practice,  but  which  makes  the  derivation  of 
the  measure  more  explicit.  Starting  from  Shannon's  assumption 
that  the  "language"  or  source  of  message  is  known  to  the  receiver, 
it  is  clear  that  disclosure  of  a  symbol  in  C  affects  the  pro¬ 
bability  of  the  succeeding  symbol  in  C  . 
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AN  ALTERNATE  METHOD 


Cl 

C2 

Gain  in  p 
of  Symbol  2 

'■  5? 

Ratio 
of  Gain 

III 

PiPj 

Information 

Gain 

log2  PL1 

PiPJ 

A 

B 

0.6 

to  1.0 

10:6 

+  0.737 

B 

A 

0.4 

to  0.5 

5:4 

+  0.322 

B 

0.6 

to  0.5 

5:6 

-  0.263 

C 

A 

0.4 

to  0.5 

5:4 

+  0.322 

B 

0.6 

to  0.5 

5:6 

-  0.263 

SUM 

0.855 

MEAN 

0.171  *  I(l;2) 

FIGURE  7 


The  gains  are  from  the  overall  probability  of  such  «.  symbol 
occurring  in  C2  to  the  probability  of  its  occurring  as  a  successor 
to  a  given  symbol  in  Cj.  1(1:2)  is  the  average  gain  of  information 
about  C2,  given  Cj,  and  indicates  the  extent  to  which  conditional 
probabilities  hold  between  the  two  sets  of  symbols.  It  is  sym¬ 
metrical,  as  can  be  tested  by  calculating  Information  about  C}, 
given  C2.  The  average  value  is  always  equal  to  or  greater  than  zero, 
but  among  the  item  values  from  which  it  is  derived,  some  may  be  nega¬ 
tive,  as  where  by  the  disclosure  of  B  in  Cj,  B  in  C2  is  made  less 
probable  than  its  a  priori  probability  of  0.6. 
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The  symbolization  of  this  value  of  "I"  derives  from  its 
interpretation  as  transmitted  information  or  I  (IN;  OUT)  in 
communication  theory,  where  the  emphasis  is  on  the  consistency 
of  relationship  between  the  signal  put  into  a  communication 
channel  and  that  recorded  by  the  receiver,  When  there  is 
complete  consistency, 

H  (IN)  *  H  (OUT)  -  H  (IN;  OUT)  »  I  (IN;  OUT) 
and  all  the  input  information  is  transmitted.  Here,  there  is  a 
one-to-one  correspondence  between  transmitted  and  received  signals. 
At  the  opposite  limit,  when  each  signal  shows  equal  probability 
of  being  recorded  either  as  itself  or  as  any  other  signal  from 
the  source,  H  (IN)  +  H  (OUT)  -  H  (IN;  OUT),  and  I  (IN;  OUT)  »  0. 

It  should  be  noticed  that  I  depends  only  on  consistency  of 
relationship.  I  is,  however,  independent  of  direction  of 
relationship  and,  in  general,  is  best  regarded  as  a  measure  of 
association  or  correlation  between  two  sets  of  categories.  It  has 
an  advantage  over  statistical  correlation  measures  in  that  no 
ordering  of  the  categories  is  required. 

It  is  convenient,  in  many  cases,  to  think  of  entropies  in 
terms  of  areas,  as  shown  in  Figure  3.  This  is  similar  to 
representing  sets  and  subsets  as  areas  using  the  Venn  diagram. 

In  Figure  8,  Elements  of  Information  Transfer,  H  (IN) 
is  represented  by  H  (X)  in  the  Venn  diagram.  H  (CUT)  is  represented 
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POTENTIAL  INFORMATION 
OF  TOTAL  SYSTEM: 

HOC,  Y) 

the  elements  of  information  transfer 


by  H  (Y).  H  (IN;  OUT)  is  represented  by  II  (X,  Y),  and  I  (IN;  OUT) 
by  I  (X,  Y). 


A  Summary  of  Communication  System  Relations 
Figure  8  serves  to  summarize  the  relations  between 

H  (X)  a  priori  uncertainty  about  source  state  x 

H  (Y)  a  priori  uncertainty  about  destination  state  y 

H  (Y/X)  measure  of  uncertainty  about  y  if  x  is  known 

H  (X/Y)  measure  of  uncertainty  about  x  if  y  is  known 

I  (X;  Y)  average  mutual  information  associated  with  x  and 


y 

H  (X,  Y)  total  average  information  in  x  and  y 
For  example:  . 

H  (X)  -  H  (X/Y)  +  I  (X;  Y)  (5) 

H  (Y)  -  H  (Y/X)  +  I  (X;  Y)  (6) 

'  H  (X,  Y)  -  H  (X)  +  H  (Y)  -  I  (X;  Y)  (7) 

Compare  with  Equation  (1). 

Thus,  having  calculated  the  values  of  H  (X),  H  (Y) ,  and 
H  (X,  Y) ,  I  (X;  Y)  may  be  calculated  from  Equation  (7),  H  (X/Y) 
from  Equation  (5),  and  H  (Y/X)  from  Equation  (6). 

Interpretation  of  System  Relations  for  this  Study 
With  reference  to  Figure  8,  this  study  is  concerned  with 

H  (X)  average  potential  information  at  the  source, 
in  bits  per  state, 
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H  (Y)  average  potential  information  at  the  destina¬ 
tion,  in  bits  per  state 
H  (X/Y)  (conditional  entropy  of  X,  given  Y) 

a  measure  of  average  failure  to  recover  informa¬ 
tion  about  the  source,  where  the  received 
category  is  known  or  accessible  to  direct 
observation,  in  bits  per  state;  also  known  as 
equivocation  (thus,  it  is  the  ambiguity  in 
interpreting  the  known  states  of  Y  as  to  what 
states  of  X  caused  them) . 

H  (Y/X)  (conditional  entropy  of  Y,  given  X). 

a  measure  of  average  failure  to  recover  infor¬ 
mation  as  to  the  receiving  category,  where 
the  transmitted  category  is  known  or  accessible 
to  direct  observation,  in  bits  per  state;  also 
known  as  noise  (thus,  it  is  the  ambiguity  in 
interpreting  the  known  states  of  X  as  to  what 
states  of  Y  they  would  produce) . 

H  (X,Y)  average  uncertainty  of  the  entire  communication 
system  as  a  whole,  in  bits  per  state, 

I  (X;Y)  (mutual  information)  a  measure  of  success  in 

recovery  of  information  about  the  source;  also, 
a  measure  of  success  in  recovery  of  information 
about  the  destination,  in  bits  per  state. 
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One  ocher  consideration  is  worth  noting  at  this  point.  The 
number  of  states  of  X  may  be  less  numerous  (coarser)  than  those 
of  Y,  or  more  numerous  (finer)  than  those  of  Y.  If  Y  is  finer 
(has  more  states)  than  X,  it  can  give  no  more  information  on  X 
than  X  itself.  If  Y  is  coarser  (has  fewer  states)  than  X,  it  will 
give  less  information  than  X,  since  the  probability  of  any  state 
x  must  be  smaller  than  the  corresponding  state  y,  even  via  a  noise- 
lesf  channel.  This  conclusion  reflects  the  same  result  as  the 
discussion  about  I  (1;  2)  following  Equation  (4).  There: 


H 

(1) 

- 

H 

(x)  - 

1.522 

(3 

states) 

H 

(2) 

H 

00  - 

0.971 

(2 

states) 

(coarser 

than  X) 

I 

(l; 

2) 

m 

I  (X; 

Y)  -  0.173 

I 

(X; 

i) 

< 

H  (Y) 

For  optimum  transmission,  there  should  be  one-to-one  correspondence 
between  the  states  and  y^. 
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HI.  ’'’’HE  RESEARCH- rO-TECHNOLQGY  COMMUNICATION  CHANNEL 


With  the  development  of  the  mathematical  model  for  a  discrete 
communication  system,  it  is  possible  to  use  data  from  an  Office  of 
Aerospace  Research  study  to  calculate  the  properties  of  the 
information  channel  from  research  to  technology. 

The  Office  of  Aerospace  Research  (OAR)  has  had  a  continuing 

interest  in  the  determination  of  patterns  of  relevance  for  Air  Force 

research.  In  a  study  concluded  during  Calendar  Year  1967,  the 

HQ  OAR  Directorate  of  Programs  identified  all  individual  research 

work  unit  records  in  each  Defense  Research  Sciences  subelement 

according  to  their  predicted  relevance  to  the  technologies  defined 

in  the  Air  Force  Systems  Command  Technical  Objective  Documents 

(TOD's).  Each  work  unit  relevant  to  a  particular  TOD  was  collected 

into  a  separate  report  once  the  determination  of  relevance  was 

completed.  In  some  cases,  a  research  work  unit  was  found  to  be 

relevant  to  more  than  one  TOD  and  so  was  counted  again.  This 

* 

work  was  conducted  under  Colonel  John  R.  Fowler.  Colonel  Fowler 
was  then  serving  as  Deputy  Chief  of  Staff  for  Plans  and  Programs 
in  OAR. 

This  study  has  employed  results  from  this  previous  effort  to 
define  a  research-technology  work  unit  matrix  as  follows.  The  Fowler 
study  produced  one  volume  of  research  work  units  classified  by  scientific 
discipline  for  each  technology  category.  The  data  abstracted  from 
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each  one  of  these  38  volumes  was  recorded  in  terms  of  numbers 
of  distinct  research  work  units  for  each  scientific  discipline 
that  was  to  be  ultimately  relevant  to  a  technology  category.  The 
collection  of  this  data  from  14  research  categories  for  each  of 
38  technology  categories  in  matrix  form  is  contained  in  Table  1, 

(a),  (b),  (c),  and  (d) .  A  total  of  3,224  work  units  is  tabulated. 

In  this  way,  transmitting  or  source  states  were  identified 
by  using  research  categories  devised  by  the  Office  of  the  Secretary 
of  Defense,  Director  of  Defense  Research  and  Engineering,  for  use 
by  the  military  services.  Receiving  or  destination  states  were 
devised  by  the  Air  Force  Systems  Command  technology  categories. 

Time  of  transmittal  is  the  year  1967.  Time  of  receipt  as  shown 
by  use  of  the  information  is  estimated  as  1967  plus  ten  years, 
on  the  average. 

The  work  unit  matrix  was  transformed,  as  in  Section  II,  to 
a  joint  probability  matrix.  This  was  done  by  dividing  each  matrix 
element  by  the  total  number  of  elements,  3,224.  The  resulting 
joint  probability  matrix  appears  in  Table  2  (a),  (b) ,  (c) ,  and  (d). 
The  joint  information  matrix  results  from  calculation  of  the  quantity 
H  *  -p  log  p^  for  all  elements  of  the  joint  probability  matrix 
as  discussed  in  Section  II.  Thus,  the  channel  characteristics  can 
be  calculated  for  this  communication  system. 
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IV. 


THE  TECHNOLOGY-TO-S YSTEMS  COMMUNICATION  CHANNEL 


A  useful  extension  of  the  analysis  would  be  the  calculation 
of  communication  channel  pioperties  for  technology-to-systems 
communication.  For  this  application  the  Technical  Objective 
Document  categories  serve  as  the  technology  items.  I  ch-rne  a 
set  of  categories  defined  by  Headquarters,  US  Air  Force,  as  the 
system  identifiers.  They  came  from  a  1969  planning  document, 

USAF  Planning  Concepts.  These  so-called  US  Air  Force  Technological 
Horizon  areas  consist  of  a  mutually  exclusive  and  exhaustive  list 
of  Air  Force  system  or  functional  ai.-as.  Since  I  treated  the 
areas  as  mutually  exclusive  and  exhaustive,  it  then  remained  only 
to  assign  the  total  work  unit  contribution  of  a  technology  category 
to  one  of  the  six  system  categories.  The  work  unit  matrix 
resulting  from  this  procedure  appears  in  Table  4.  This  matrix 
next  appears  transformed  to  a  joint  probability  matrix  in  Table  5. 
Table  6  contains  the  joint  information  matrix. 
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r.  THE  RESEARCH-TO-SYSTEMS  COMMUNICATION  CHANNEL 

This  channel  uses  the  same  source  data  that  the  previous 
cases  have  used.  The  same  research  categories  appear.  Again,  I 
used  the  identical  system  categories  as  before.  Research 
contributors  to  each  technology  were  identified  from  Table  1. 
Regrouping  and  adding  totals  of  research  work  units  for  each 
system  category  by  working  back  from  the  Technology-to-Systens 
Table  4  produced  a  Research-to-Systems  Work  Unit  Matrix, 

Table  7. 

An  example  of  the  procedure  may  be  helpful.  Take  the  case 
of  the  Technological  Horizons  area  of  Weaponry.  From  Table  4, 
the  Technology-to-Systems  Work  Unit  Matrix,  Weaponry  has 
contributions  from  technology  categories  as  follows: 


TOD  Number 

Work  Units 

68-1 

26 

68-7 

238 

68-8 

24 

288 

Using  Table  1,  the  Research-Technology  Work  Unit  Matrix,  one  can 
list  the  number  of  work  units  in  each  Defense  Research  Sciences 
category  contributing  to  eafch  of  the  technology  categories  as 
shown  in  Figure  9. 
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RESEARCH  EFFORT  COHTRIBUIIJNS  TO  WEAPONRY 


Adv  Weapons 

Chem-Bio 

Munitions 

Conventional 

Munitions 

Total 

68-1 

68-7 

68-8 

0 

DRS  Support 

1 

1 

2 

1 

Gen  Physics 

1 

1 

2 

3 

Chemistry 

16 

207 

16 

239 

4 

Math  Sciences 

3 

1 

4 

7 

Mechanics 

1 

1 

4 

6 

8 

Energy  Conversion 

4 

3 

7 

12 

Bio  &  Medical  Sciences 

27 

1 

28 

Total 

288 

FIGURE  9 


The  "total"  column  appears  in  Table  7.  The  same  procedure,  then, 
has  been  used  to  produce  all  other  columns  in  Table  7. 

The  associated  joint  probability  matrix  appears  in  Table  8 


and  the  joint  information  matrix  in  Table  9. 


VI. 


ASSUMPTIONS 


First,  the  work  may  be  done  by  one  individual  or  bv  a  team, 
but,  for  simplicity,  one  technical  report  is  assumed  to  result 
from  one  work  unit  per  year.  The  use  of  the  so-called  "work  unit" 
is  important  in  this  analysis,  and  therefore  calls  for  a  defini¬ 
tion  of  the  term.  Normally,  a  work  unit  is  the  lowest  integral 
technical  effort  which  is  defined  in  research  or  technology.  It 
is  the  quantum  of  organized  research  effort  in  that  a  report  will 
be  produced  as  a  result  of  the  effort  of  one  or  more  participants 
toward  a  common  goal.  In  Air  Force  research  and  development 
documentation,  it  is  a  single  contractual  effort  or  a  single 
identified  in-house  laboratory  research  activity. 

Second,  each  report  from  each  work  unit  is  assumed  to 
represent  the  same  amount  of  information  per  work  unit.  Reference 
4  gives  an  estimate  of  the  information  content  of  an  English 
language  word  as  11.82  bits/word  for  the  average  word  of  5.5  letters. 
This  figure  takes  into  account  the  redundancy  of  the  English  language. 
Assuming  that  the  average  report  (from  a  small  sample)  is  10,638 
words  long,  such  a  report  would  contain  10,638  x  11.82  ■  125,741.16 
bits  per  report.  Since  we  have  assumed  one  report  per  work  unit, 
this  number  of  bits  per  report  is  also  the  number  of  bits  per  year 
per  work  unit.  (A  total  of  3,224  work  units  per  year  would  indicate 
125,741.16  x  3,224  ■  405,389,499.84  potential  bits  per  year.  I 
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calculate  this  quantity  only  for  purposes  of  demonstrating  how 
one  might  calculate  the  potential  information  flow  with  more 
reliable  estimates  on  actual  bits  per  report  and  number  of 
reports  per  year. )  The  assumption  of  equal  amounts  of  information 
(one  report)  from  each  work  unit  allows  use  of  a  work  unit  matrix 
to  establish  the  characteristics  of  the  communication  channel  in 
each  case. 

Third,  time-lag  between  origin  and  use  of  a  research  or 
technology  result  is  assumed  not  to  affect  the  characteristics  of 
the  several  communication  channels  discussed  in  this  study.  Basic 
to  the  calculations  is  the  assumption  that,  if  the  results  occur 
at  some  time  t0,  then  at  some  later  time,  tQ  +  T1»  research 

results  would  have  been  applied,  and  that  at  a  subsequent  time 
(tQ  +  T  )  +  T  ,  all  technology  results  would  have  been  applied. 
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VII.  RESULTS 


Figure  10  shows  the  results  of  calculations  using  joint 
information  matrices  for  the  three  communication  channels. 

In  each  case,  the  "X"  quantity  serves  to  identify  the  originating 
category,  while  "Y"  serves  to  identify  the  receiving  category. 

H  (X)  is  obtained  in  each  case  by  summing  the  marginal  totals  of 
the  originating  quantity.  On  Table  3  (d),  this  quantity  is  labeled 
H  (R)  and  the  total  is  indicated  at  the  end  of  the  H  (R)  row  as 
3.2733  (bits/research  category).  This  amounts  to 
H  (R)  ■*  -I  p±  log2 

H  (Y)  is  obtained  by  summing  the  marginal  totals  of  the  receiving 
quantity.  This  Indicates  the  operation  H  (T)  *  -E  p^  log^p^. 

The  total  appears  on  Table  3  (d)  as  4.4400  under  the  column  H  (T) . 

H  (X,  Y)  results  from  adding  all  matrix  elements  by 
row  and  then  adding  these  totals  together  as  stated  by 


H  (X,  Y)  -  -E  E  p  log  p  , 
i  j  J  2  1J 

since  each  element  of  the  matrix  in  Table  3  (d)  represents  an 


elementary  p .  leg  p.,. 

2  XJ 

In  Figure  10,  beginning  with  the  basic  quantities  H  (X), 

E  (Y) ,  and  H  (X,  Y) ,  other  remaining  parameters  may  be  calculated 
as  shown  on  page  34. 
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I  (X;  Y)  -  H  (X)  +  H  (Y)  -  H  (X,  Y) 

H  (X/Y)  -  H  (X)  -  I  (X;  Y) 

H  (Y/X)  =  H  (Y)  -  I  (X;  Y) 

Figure  11  provides  the  data  needed  to  calculate  these 
probabilities : 

1.  Probability  of  tracing  research  origins  of  technology. 

2.  Probability  of  tracing  technology  origins  of  systems. 

3.  Probability  of  tracing  research  origins  of  systems. 

It  also  provides  data  for  these  added  probability  calcula¬ 
tions  : 

4.  Probability  of  forecasting  applications  of  research  in 
technology . 

5.  Probability  of  forecasting  applications  of  technology 
in  systems. 

6.  Probability  of  forecasting  applications  ox  research  in 
systems. 

To  get  the  firsc  three  probabilities  above,  recall  that  the 
quantity  H  (X/Y)  has  been  previously  interpreted  as  a  measure 
of  failure  to  recover  information  about  the  source  where  the 
received  category  is  known.  Previous  discussion  also  showed  that 
I  (X;Y)  can  be  interpreted  as  a  symmetrical  quantity  for  both  source 
and  received  category  as  follows:  mutual  information;  a  measure 
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Probability  of  Success  in  Tracing  Origins  Probability  of  Success  in  Forecasting  Applications 
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PROBABILITIES  OF  SUCCESSFUL  IDENTIFICATION 


of  success  in  recovery  of  information  about  the  source  where 
the  received  category  is  known  or  of  success  in  recovery  of 
information  about  the  receiving  category  where  the  source  cate¬ 
gory  is  known. 

I (X‘Y) 

The  ratio  represents  the  fraction  of  information 

available  at  the  source  successfully  recovered,  where  the 
received  category  is  known.  Hence,  it  ioay  be  interpreted  as  the 
probability  of  tracing  (successfully  identifying)  origins  of  the 
receiving  category.  The  ratio  '  likewise,  represents  the 

fraction  of  information  about  the  receiving  category  successfully 
recovered,  where  the  source  category  is  known.  Therefore,  I 
interpret  it  as  the  probability  of  forecasting  (successfully 
Identifying)  applications  of  the  source  category.  Figure  11  shows 


these  results. 


VIII.  OBSERVATIONS 


Certain  general  observations  can  be  made  about  the 
communication  channels  in  the  research-technology-systems 
area: 

1.  There  are  fundamental  limitations  which  are  imposed 
on  communication  channel  effectiveness  by  the  way  in  which 
the  source  and  receiving  categories  are  defined  or  "packaged." 
That  is,  one  should  be  compatible  with  the  other  by  having  a 
one-to-one  correspondence  in  terms  of  (a)  numbers  of  categories 
into  which  each  is  divided  and  (b)  relevance  of  each  source 
category  to  as  few  of  the  destination  categories  as  possible 
(preferably  to  only  one).  By  the  nature  of  broadly  relevant 
research  categories  which  inherently  support  more  than  one 
technology  category,  limitations  exist  in  both  tracing  and 
forecasting  in  the  channels  analyzed.  Ideally,  for  38 
technology  categories  there  should  exist  38  res*  vrch  categories 
with  one-to-one  correspondence.  This  would  permit  perfect 
forecasting  or  tracing  of  origins.  In  the  real  world,  a  100Z 
probability  of  success  is  not  possible  because  of  the  broad 
relevance  of  each  category  of  basic  science  to  several  modern 
technology  categories,  as  may  be  seen  in  Table  1. 

All  of  these  factors  have  not  been  lost  on  research 
and  technology  planners  during  1969  -  1970,  as  indicated  by  the 
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extensive  effort  under  way  during  this  period  to  "repackage"  both 
research  and  technology  into  more  numerous  new  categories  each 
having  more  obvious  relevance  to  a  military  function  or  operation. 

2.  These  results  may  be  helpful  in  research  and  technology 
planning.  They  may  be  interpreted  to  indicate  probability  of 
success  for  forecasting  relevant  tasks  in  the  exploratory  or  in 
the  normative  directions  (see  Figure  11) : 

Exploratory  (Applications)  Normative  (Origins) 


Research  to  Technology 

.3864 

Technology  to  Research 

.5242 

Technology  to  Systems 

1.0000 

Systems  to  Technology 

.5027 

Research  to  Systems 

.3841 

Systems  to  Research 

.2619 

Of  course,  these  results  are  c’erived  in  the  context  of  the  assump¬ 
tions  and  in  the  contest  of  the  research,  technology,  and  system 
categories  used  in  this  study.  Any  modification  of  either  context 
could  alter  these  results. 

3.  These  results  give  some  Insight  into  efforts  to  Improve 
the  research  and  development  planning  process  by  studying  past 
contributions  of  research  and  technology  to  military  weapons  and 
adjusting  current  efforts  acc<  rdingly.  The  probability  of  success 
of  identifying  the  origins  of  such  efforts  from  Figure  11  is: 

Technology  Origins  of  Sy items  .5027 

-Research  OrUiav  jf  Sys tens*  .2619 

Project  Hindsight  (1969)  is  an  example  of  such  a  study. 
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IX. 


FUNDAMENTAL  LIMITATIONS  IN  ELECTRICAL  COMMUNICATION 


In  the  design  of  a  communication  system  there  are  two  kinds 
of  constraints.  To  begin  with  there  are  the  technological  pro¬ 
blems  presented  by  the  engineering  facts  of  life.  There  are  also 
the  fundamental  physical  limitations  imposed  by  the  laws  of 
nature.  Technological  questions  aside,  it  is  the  fundamental 
limitations  which  ultimately  determine  what  can  or  cannot  be 
accomplished.  The  fundamental  limitations  in  electrical  communi¬ 
cations  currently  identified  are  bandwidth  and  noi3e. 

Bandwidth 

Bandwidth  is  the  width  of  the  frequency  spectrum  of  the 
signals  or  messages  in  a  communication  system.  It  is  a  measure 
of  the  signal  or  message  rate.  Similarly,  the  rate  at  which  a 
system  can  change  stored  energy  is  given  by  its  frequency  response, 
measured  In  terms  of  the  system  bandwidth.  Transmitting  a  large 
amount  of  information  in  a  small  amount  of  time  requires  wideband 
signals  to  represent  the  information  and  wideband  systems  to 
accommodate  the  signals  (Reference  5). 

Noise 

Relative  to  the  noise  limitation,  successful  electrical  com¬ 
munication  depends  on  how  accurately  the  receiver  can  determine 
which  signal  was  actually  sent,  as  distinguished  from  signals  that 
might  have  been  sent.  Noise  is  always  present  in  electrical 
systems,  and  it  limits  our  ability  to  correctly  identify  the 
intended  signal  and  therefore  limits  communication  of  information. 
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In  ordinary  electrical  systems  under  normal  conditions,  the 
signal-to-noise  ratio  is  large  enough  for  noise  effects  to  be 
insignificant . 

In  the  final  analysis,  given  a  system  oi  fixed  bandwidth 
and  signal- to-noise  ratio,  there  is  a  definite  upper  limit  on  the 
rate  at  which  information  can  be  transmitted  by  that  system. 

Inis  upper  limit  is  called  the  information  capacity  and  is  one  of 
the  central  concepts  of  information  theory  (Reference  6). 

Discrete  Channels 

A  discrete  channel  is  one  which  transmits  information  by 
successively  assuming  various  disjoint  electrical  states.  The 
capacity  of  a  discrete  channel  depends  on  signaling  speed,  and 
the  number  of  states.  The  communication  systems  analyzed  in  this 
study  have  been  discrete  channels. 

The  capacity  equation  (Reference  7)  for  a  discrete  channel 
with  a  flat  power  spectrum  (uniform  probability  density  function) 
for  x  is: 


C 


ITWlog  S 

T  V  N 


'  (bits/sec) 


where 


(8) 


2W  ■  Total  number  of  samples/f ec  permitted  by  the 

Nyquist  sampling  rate,  2W,  which  is  the  maxi¬ 
mum  sampling  frequency. 

2TW  •  Total  number  of  samples  (in  the  time  interval, 

T  second) 
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2TW 

~T 


Samples  per  second 
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Fundamental  Limitations 


Hence,  a  fundamental  limitation  in  tracing  and  forecasting 
exists  for  channels  in  the  research  -  technology  -  systems  area, 
because  cn  is  not  zero.  For  these  channels,  a  source  category  is 
relevant  in  many  cases  to  more  than  one  destination  category 
(average  on  ^  0) ,  and  the  number  of  source  categories  does  not 
match  the  number  of  destination  categories  ov  ±  cf  .  These  limita- 
tions  prevail  because  of  the  fundamentally  broad  relevance  of 
research  to  technology  and  to  systems.  In  Air  Force  research  and 
development  planning  (and  all  mission-oriented  planning) ,  improve¬ 
ment  in  tracing  and  forecasting  may  be  made  by  changing  the  band¬ 
width  (source  categories)  to  better  match  the  receiving  categories, 
and  by  increasing  signal-to-noise  power  ratio  (increasing  the 
number  of  distinguishable  receiving  categories).  Channel  noise 
places  fundamental  limitations  on  the  rate  but  not  the  accuracy  of 
information  transfer. 

The  noisy  coding  theorem  of  Shannon  sets  an  upper  limit  (the 
cnannel  capacity)  on  the  rate  of  information  transfer  but  guarantees 
the  existence  of  codes  that  allow  transmission  at  this  rate  with 
arbitrarily  9mall  probability  of  error.  Improvements  suggested  in 
this  and  earlier  sections  would  change  each  channel  to  give 
imoroved  channel  capacity  and,  hence,  improved  tracing  and  planning 
capability. 
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RESEARCH-TECHNOLOGY  JOINT  INFORMATION  MATRIX  (in  Bits),  PART  1 

TABLE  3  (a) 


UK FENS E  RESEARCH  SCIENCES 
(RESEARCH  WORK  UNIT  CATEGORIES) 


DEFENSE  RESEARCH  SCIENCES 


RESEARCH-TECHNOLOGY  JOINT  INFORMATION  MATRIX,  Part 
TABLE  3  (c) 


DEFENSE  RESEARCH  SCIENCES 
(RESEARCH  WORK  UNIT  CATEGORIES) 
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TECHNOLOGY-SYSTEMS  WORK  UNIT  MATRIX,  PART  1 
TABLE  4  (a) 


USAF  TECHNOLOGICAL  HORIZON  AREAS 
(SYSTEM  CATEGORIES) 
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TECHNOLOGY-SYSTEMS  WORK  UNIT  MATRIX,  PART  2 


USAF  TECHNOLOGICAL  HORIZON  AREAS 
(SYSTEM  CATEGORIES) 


TECHNOLOGY-SYSTEMS  WORK  UNIT  MATRIX,  PART  3 
TABLE  4  (c) 


USAF  TECHNOLOGICAL  HORIZON  AREAS 
(SYSTEM  CATEGORIES) 


TECHNOLOGY-SYSTEMS  JOINT  PROBABILITY  MATRIX,  PART  1 
TABLE  5  (a) 
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TECHNOLOGY-SYSTEMS  JOINT  PROBABILITY  MATRIX,  PART  2 
TABLE  5  (b) 
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(SYSTEM  CATEGORIES) 


RESEARCH-SYSTEMS  WORK  UNIT  MATRIX 

TABLE  7 
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TABLE  8 
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(SYSTEM  CATEGORIES) 


RESEARCH- SYSTEMS  JOINT  INFORMATION  iuVTRIX 
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